{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. 回归\n",
    "\n",
    "- 往往为多输入，单输出"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. 分类\n",
    "\n",
    "- 通常为多输出\n",
    "- 输出i为预测为第i类的概率"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 分类问题的标签\n",
    "\n",
    "- 一位有效编码（one-hot）：每一种类别对应一位，正确类别对应位为1，其余位为0\n",
    "\n",
    "## 2.2 分类问题的输出\n",
    "\n",
    "- 比如预测n个类别，每个类别的置信度为$c_i$，则需要将这n个类别的置信度经过一个softmax函数，使类别与类别之间的取值更具差异化，使大的更大，小的更小"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 分类问题的损失函数\n",
    "\n",
    "- 常用交叉熵损失函数\n",
    "  \n",
    "  ![image-20230315095953261](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20230315095953261.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. 分类问题的数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fashion-MNIST数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "from torch.utils import data\n",
    "from torchvision import transforms\n",
    "from d2l import torch as d2l\n",
    "d2l.use_svg_display()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 将图像数据从PIL图像类型数据转为tensor类型\n",
    "  ```py\n",
    "    # 定义一系列转换操作\n",
    "    transform = transforms.Compose([\n",
    "        transforms.ToTensor(),\n",
    "        transforms.Normalize((0.5,), (0.5,))\n",
    "    ])\n",
    "  ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "trans = transforms.ToTensor()\n",
    "mnist_train = torchvision.datasets.FashionMNIST(root='../data', train=True, transform=trans, download=True)\n",
    "mnist_test = torchvision.datasets.FashionMNIST(root='../data', train=False, transform=trans, download=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- mnist_train是一个数据集对象，可以像列表一样使用，如：\n",
    "  - 取第一个样本：`mnist_train[0]`\n",
    "  - 取第一个样本的feature：`mnist_train[0][0]`\n",
    "  - 取第一个样本的label：`mnist_train[0][1]`\n",
    "- 每个样本的feature是一个28*28的图像"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Tensor, torch.Size([1, 28, 28]))"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(mnist_train[0][0]), mnist_train[0][0].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [实验1]：使用不同进程数，比较读取相同数据集的速度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在使用`DataLoader`制作数据集时，可以制定参数`num_workers`设置读取数据的进程数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<BarContainer object of 8 artists>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"391.245pt\" height=\"297.190125pt\" viewBox=\"0 0 391.245 297.190125\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-07-13T20:38:48.370600</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 297.190125 \n",
       "L 391.245 297.190125 \n",
       "L 391.245 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 26.925 273.312 \n",
       "L 384.045 273.312 \n",
       "L 384.045 7.2 \n",
       "L 26.925 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 43.157727 273.312 \n",
       "L 76.455629 273.312 \n",
       "L 76.455629 19.872 \n",
       "L 43.157727 19.872 \n",
       "z\n",
       "\" clip-path=\"url(#pa991f5177b)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 84.780105 273.312 \n",
       "L 118.078007 273.312 \n",
       "L 118.078007 80.808185 \n",
       "L 84.780105 80.808185 \n",
       "z\n",
       "\" clip-path=\"url(#pa991f5177b)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 126.402483 273.312 \n",
       "L 159.700385 273.312 \n",
       "L 159.700385 96.142503 \n",
       "L 126.402483 96.142503 \n",
       "z\n",
       "\" clip-path=\"url(#pa991f5177b)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 168.02486 273.312 \n",
       "L 201.322762 273.312 \n",
       "L 201.322762 97.808892 \n",
       "L 168.02486 97.808892 \n",
       "z\n",
       "\" clip-path=\"url(#pa991f5177b)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_7\">\n",
       "    <path d=\"M 209.647238 273.312 \n",
       "L 242.94514 273.312 \n",
       "L 242.94514 98.83537 \n",
       "L 209.647238 98.83537 \n",
       "z\n",
       "\" clip-path=\"url(#pa991f5177b)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_8\">\n",
       "    <path d=\"M 251.269615 273.312 \n",
       "L 284.567517 273.312 \n",
       "L 284.567517 91.262665 \n",
       "L 251.269615 91.262665 \n",
       "z\n",
       "\" clip-path=\"url(#pa991f5177b)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_9\">\n",
       "    <path d=\"M 292.891993 273.312 \n",
       "L 326.189895 273.312 \n",
       "L 326.189895 84.551629 \n",
       "L 292.891993 84.551629 \n",
       "z\n",
       "\" clip-path=\"url(#pa991f5177b)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_10\">\n",
       "    <path d=\"M 334.514371 273.312 \n",
       "L 367.812273 273.312 \n",
       "L 367.812273 80.332072 \n",
       "L 334.514371 80.332072 \n",
       "z\n",
       "\" clip-path=\"url(#pa991f5177b)\" style=\"fill: #1f77b4\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <defs>\n",
       "       <path id=\"ma7713e503e\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma7713e503e\" x=\"59.806678\" y=\"273.312\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 1 -->\n",
       "      <g transform=\"translate(56.625428 287.910437) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma7713e503e\" x=\"101.429056\" y=\"273.312\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(98.247806 287.910437) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma7713e503e\" x=\"143.051434\" y=\"273.312\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 3 -->\n",
       "      <g transform=\"translate(139.870184 287.910437) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
       "Q 3050 2419 3304 2112 \n",
       "Q 3559 1806 3559 1356 \n",
       "Q 3559 666 3084 287 \n",
       "Q 2609 -91 1734 -91 \n",
       "Q 1441 -91 1130 -33 \n",
       "Q 819 25 488 141 \n",
       "L 488 750 \n",
       "Q 750 597 1062 519 \n",
       "Q 1375 441 1716 441 \n",
       "Q 2309 441 2620 675 \n",
       "Q 2931 909 2931 1356 \n",
       "Q 2931 1769 2642 2001 \n",
       "Q 2353 2234 1838 2234 \n",
       "L 1294 2234 \n",
       "L 1294 2753 \n",
       "L 1863 2753 \n",
       "Q 2328 2753 2575 2939 \n",
       "Q 2822 3125 2822 3475 \n",
       "Q 2822 3834 2567 4026 \n",
       "Q 2313 4219 1838 4219 \n",
       "Q 1578 4219 1281 4162 \n",
       "Q 984 4106 628 3988 \n",
       "L 628 4550 \n",
       "Q 988 4650 1302 4700 \n",
       "Q 1616 4750 1894 4750 \n",
       "Q 2613 4750 3031 4423 \n",
       "Q 3450 4097 3450 3541 \n",
       "Q 3450 3153 3228 2886 \n",
       "Q 3006 2619 2597 2516 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-33\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma7713e503e\" x=\"184.673811\" y=\"273.312\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 4 -->\n",
       "      <g transform=\"translate(181.492561 287.910437) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma7713e503e\" x=\"226.296189\" y=\"273.312\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 5 -->\n",
       "      <g transform=\"translate(223.114939 287.910437) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma7713e503e\" x=\"267.918566\" y=\"273.312\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 6 -->\n",
       "      <g transform=\"translate(264.737316 287.910437) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
       "Q 1688 2584 1439 2293 \n",
       "Q 1191 2003 1191 1497 \n",
       "Q 1191 994 1439 701 \n",
       "Q 1688 409 2113 409 \n",
       "Q 2538 409 2786 701 \n",
       "Q 3034 994 3034 1497 \n",
       "Q 3034 2003 2786 2293 \n",
       "Q 2538 2584 2113 2584 \n",
       "z\n",
       "M 3366 4563 \n",
       "L 3366 3988 \n",
       "Q 3128 4100 2886 4159 \n",
       "Q 2644 4219 2406 4219 \n",
       "Q 1781 4219 1451 3797 \n",
       "Q 1122 3375 1075 2522 \n",
       "Q 1259 2794 1537 2939 \n",
       "Q 1816 3084 2150 3084 \n",
       "Q 2853 3084 3261 2657 \n",
       "Q 3669 2231 3669 1497 \n",
       "Q 3669 778 3244 343 \n",
       "Q 2819 -91 2113 -91 \n",
       "Q 1303 -91 875 529 \n",
       "Q 447 1150 447 2328 \n",
       "Q 447 3434 972 4092 \n",
       "Q 1497 4750 2381 4750 \n",
       "Q 2619 4750 2861 4703 \n",
       "Q 3103 4656 3366 4563 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_7\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma7713e503e\" x=\"309.540944\" y=\"273.312\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 7 -->\n",
       "      <g transform=\"translate(306.359694 287.910437) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-37\" d=\"M 525 4666 \n",
       "L 3525 4666 \n",
       "L 3525 4397 \n",
       "L 1831 0 \n",
       "L 1172 0 \n",
       "L 2766 4134 \n",
       "L 525 4134 \n",
       "L 525 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-37\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_8\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma7713e503e\" x=\"351.163322\" y=\"273.312\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 8 -->\n",
       "      <g transform=\"translate(347.982072 287.910437) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
       "Q 1584 2216 1326 1975 \n",
       "Q 1069 1734 1069 1313 \n",
       "Q 1069 891 1326 650 \n",
       "Q 1584 409 2034 409 \n",
       "Q 2484 409 2743 651 \n",
       "Q 3003 894 3003 1313 \n",
       "Q 3003 1734 2745 1975 \n",
       "Q 2488 2216 2034 2216 \n",
       "z\n",
       "M 1403 2484 \n",
       "Q 997 2584 770 2862 \n",
       "Q 544 3141 544 3541 \n",
       "Q 544 4100 942 4425 \n",
       "Q 1341 4750 2034 4750 \n",
       "Q 2731 4750 3128 4425 \n",
       "Q 3525 4100 3525 3541 \n",
       "Q 3525 3141 3298 2862 \n",
       "Q 3072 2584 2669 2484 \n",
       "Q 3125 2378 3379 2068 \n",
       "Q 3634 1759 3634 1313 \n",
       "Q 3634 634 3220 271 \n",
       "Q 2806 -91 2034 -91 \n",
       "Q 1263 -91 848 271 \n",
       "Q 434 634 434 1313 \n",
       "Q 434 1759 690 2068 \n",
       "Q 947 2378 1403 2484 \n",
       "z\n",
       "M 1172 3481 \n",
       "Q 1172 3119 1398 2916 \n",
       "Q 1625 2713 2034 2713 \n",
       "Q 2441 2713 2670 2916 \n",
       "Q 2900 3119 2900 3481 \n",
       "Q 2900 3844 2670 4047 \n",
       "Q 2441 4250 2034 4250 \n",
       "Q 1625 4250 1398 4047 \n",
       "Q 1172 3844 1172 3481 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <defs>\n",
       "       <path id=\"m2f68b70e05\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2f68b70e05\" x=\"26.925\" y=\"273.312\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(13.5625 277.111219) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2f68b70e05\" x=\"26.925\" y=\"225.991745\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(13.5625 229.790964) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2f68b70e05\" x=\"26.925\" y=\"178.67149\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 4 -->\n",
       "      <g transform=\"translate(13.5625 182.470709) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2f68b70e05\" x=\"26.925\" y=\"131.351235\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 6 -->\n",
       "      <g transform=\"translate(13.5625 135.150454) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2f68b70e05\" x=\"26.925\" y=\"84.030981\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 8 -->\n",
       "      <g transform=\"translate(13.5625 87.830199) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2f68b70e05\" x=\"26.925\" y=\"36.710726\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_14\">\n",
       "      <!-- 10 -->\n",
       "      <g transform=\"translate(7.2 40.509944) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"patch_11\">\n",
       "    <path d=\"M 26.925 273.312 \n",
       "L 26.925 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_12\">\n",
       "    <path d=\"M 384.045 273.312 \n",
       "L 384.045 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_13\">\n",
       "    <path d=\"M 26.925 273.312 \n",
       "L 384.045 273.312 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_14\">\n",
       "    <path d=\"M 26.925 7.2 \n",
       "L 384.045 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pa991f5177b\">\n",
       "   <rect x=\"26.925\" y=\"7.2\" width=\"357.12\" height=\"266.112\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "batch_size = 256\n",
    "time_spent = []\n",
    "num_process = []\n",
    "\n",
    "for i in range(8):\n",
    "    train_iter = iter(data.DataLoader(mnist_train, batch_size, shuffle=True, num_workers=i+1))\n",
    "    timer = d2l.Timer()\n",
    "    for x,y in train_iter:\n",
    "        ...\n",
    "    time_spent.append(timer.stop())\n",
    "    num_process.append(i+1)\n",
    "\n",
    "plt.bar(num_process, time_spent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [实验2]：编写一个完整的读取数据集函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([256, 1, 28, 28])\n",
      "torch.Size([256])\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "from torch.utils import data\n",
    "from torchvision import transforms\n",
    "from d2l import torch as d2l\n",
    "\n",
    "def load_data_fashion_minist(batch_size, resize = None):\n",
    "    # 预处理\n",
    "    trans = [\n",
    "        transforms.ToTensor()\n",
    "    ]\n",
    "    if resize:\n",
    "        trans.insert(transforms.Resize(resize))\n",
    "    trans = transforms.Compose(trans)\n",
    "\n",
    "    # 读取数据集\n",
    "    mnist_train = torchvision.datasets.FashionMNIST(root='../data', train=True, transform=trans, download=True)\n",
    "    mnist_test = torchvision.datasets.FashionMNIST(root='../data', train=False, transform=trans, download=True)\n",
    "\n",
    "    # 制作成dataloader\n",
    "    train_iter = data.DataLoader(mnist_train, batch_size, shuffle=True, num_workers=4)\n",
    "    test_iter = data.DataLoader(mnist_test, batch_size, shuffle=True, num_workers=4)\n",
    "\n",
    "    return train_iter, test_iter\n",
    "\n",
    "# 测试\n",
    "batch_size = 256\n",
    "train_iter, test_iter = load_data_fashion_minist(batch_size)\n",
    "print(next(iter(train_iter))[0].shape)\n",
    "print(next(iter(train_iter))[1].shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. softmax回归的简单实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1f962e67c10>]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjUAAAGdCAYAAADqsoKGAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABXR0lEQVR4nO3de1xUdf4/8NeZgZkR5I4wgFwEFe+ooIh5TUqsTd1sM1fDzMxc7SLf3Vx/32/aZb/hrpvrVm62Fmlrm677bbO0LMV7IihkXjIElJswXGW4yWVmzu8PZHQChEGGub2ej8d5bJ7r+zjNzqtzPhdBFEURRERERFZOYu4CiIiIiHoCQw0RERHZBIYaIiIisgkMNURERGQTGGqIiIjIJjDUEBERkU1gqCEiIiKbwFBDRERENsHB3AX0Fp1Oh6KiIri4uEAQBHOXQ0RERF0giiJqamrg7+8PieTuz2LsJtQUFRUhMDDQ3GUQERFRNxQUFKB///533cduQo2LiwuAlr8UV1dXM1dDREREXVFdXY3AwED97/jd2E2oaX3l5OrqylBDRERkZbrSdIQNhYmIiMgmMNQQERGRTWCoISIiIpvAUENEREQ2gaGGiIiIbAJDDREREdkEhhoiIiKyCQw1REREZBMYaoiIiMgmMNQQERGRTWCoISIiIpvAUENEREQ2wW4mtDSV7NJa/Du9EJ7Ojnh2Spi5yyEiIrJbfFJzj3LKarH1WA7+mZpv7lKIiIjsGkPNPbpvoDccpQJyK+qRW15n7nKIiIjsFkPNPeord0BUsCcA4GhmqZmrISIisl8MNT1gWng/AMDRK2VmroSIiMh+MdT0gKm3Qk1KTgUamrVmroaIiMg+MdT0gHBfFyhdFWjU6HD6aoW5yyEiIrJLDDU9QBCE26+gMvkKioiIyBwYanpIa6g5xnY1REREZsFQ00PuG+gNB4mAa+V1yKtg124iIqLexlDTQ1wUjogM9gDAV1BERETmwFDTg6aF+wDgKygiIiJzYKjpQa3tak7llLNrNxERUS/rVqjZsmULQkJCoFAoEB0djbS0tLvuX1VVhZUrV8LPzw9yuRyDBw/GV199ZdQ5GxoasHLlSnh5eaFv376YN28eSkpKulO+yQxRtnTtbmjWIe1apbnLISIisitGh5rdu3cjISEB69evR0ZGBiIiIjBz5kyUlrY/RUBTUxMeeOAB5Obm4t///jcyMzOxbds2BAQEGHXO1atX48svv8SePXtw7NgxFBUV4dFHH+3GLZuOIAiYOphdu4mIiMxCNNL48ePFlStX6v+s1WpFf39/MTExsd3933vvPTE0NFRsamrq9jmrqqpER0dHcc+ePfp9Ll++LAIQU1JSulS3Wq0WAYhqtbpL+3fXV+eLxOA1+8Tpfz5i0usQERHZA2N+v416UtPU1IT09HTExsbq10kkEsTGxiIlJaXdY7744gvExMRg5cqV8PX1xYgRI/Dmm29Cq9V2+Zzp6elobm422GfIkCEICgrq8LqNjY2orq42WHrDxIHekEoEXC2rQ0Flfa9ck4iIiIx8/VReXg6tVgtfX1+D9b6+vlCpVO0ec/XqVfz73/+GVqvFV199hVdeeQVvvfUW/vCHP3T5nCqVCjKZDO7u7l2+bmJiItzc3PRLYGCgMbfabW59HBEZ1Nq1m7N2ExER9RaT937S6XTw8fHB3//+d0RGRmL+/Pn47//+b2zdutWk1127di3UarV+KSgoMOn17jSVUyYQERH1OqNCjbe3N6RSaZteRyUlJVAqle0e4+fnh8GDB0MqlerXDR06FCqVCk1NTV06p1KpRFNTE6qqqrp8XblcDldXV4Olt9zu2s1Zu4mIiHqLUaFGJpMhMjISycnJ+nU6nQ7JycmIiYlp95j77rsP2dnZ0Ol0+nVXrlyBn58fZDJZl84ZGRkJR0dHg30yMzORn5/f4XXNaZifK3xc5LjZrMWZXHbtJiIi6g1Gv35KSEjAtm3bsGPHDly+fBkrVqxAXV0dlixZAgCIj4/H2rVr9fuvWLEClZWVePHFF3HlyhXs378fb775JlauXNnlc7q5uWHp0qVISEjAkSNHkJ6ejiVLliAmJgYTJky417+DHseu3URERL3PwdgD5s+fj7KyMqxbtw4qlQqjR4/GgQMH9A198/PzIZHczkqBgYH45ptvsHr1aowaNQoBAQF48cUXsWbNmi6fEwD+8pe/QCKRYN68eWhsbMTMmTPxt7/97V7u3aSmhftgT3ohjl0pwyvmLoaIiMgOCKIoiuYuojdUV1fDzc0NarW6V9rXqG82Y+wbB6HViTi5Zjr6eziZ/JpERES2xpjfb879ZCJufRwxNsgdAF9BERER9QaGGhNqnbWboYaIiMj0GGpMqLWx8KmccjRq2LWbiIjIlBhqTGiYnyu8+8pR36TF2dwb5i6HiIjIpjHUmJBEcmfXbk6ZQEREZEoMNSY2jVMmEBER9QqGGhObPMgbEgHIKq3F9aqb5i6HiIjIZjHUmJi7kwxjOGs3ERGRyTHU9IJpnDKBiIjI5BhqekHreDWnssvRpNF1sjcRERF1B0NNLxju7wrvvjLUNWlxNo+zdhMREZkCQ00vkEgETLn1CuoYX0ERERGZBENNL+GUCURERKbFUNNLJg9s6dqdWVKDInbtJiIi6nEMNb3Ew1mGiEB3AMCxK3xaQ0RE1NMYanrRtMGtr6A4Xg0REVFPY6jpRa1TJnyXXcGu3URERD2MoaYXjQxwg5ezDLWNGqTncdZuIiKinsRQ04vu7Np99ApfQREREfUkhppe1voKiuPVEBER9SyGml42eVA/CALwk6oGxWp27SYiIuopDDW9zNNZhoj+7gCA4+zaTURE1GMYasyg9RUURxcmIiLqOQw1ZtA6ZcLJrHI0a9m1m4iIqCcw1JjByAA3eDg5oqZRgwx27SYiIuoRDDVmIDXo2s1XUERERD2BocZM2K6GiIioZzHUmMmUW127LxdXo6S6wdzlEBERWT2GGjPx6ivHqAA3AByIj4iIqCcw1JjR1Fu9oDhlAhER0b1jqDGj1nY1J7LKoWHXbiIionvCUGNGEf3dW7p2N2iQkV9l7nKIiIisGkONGUklAiYPujXBJV9BERER3ROGGjNj124iIqKewVBjZq1Pai4VVaO0hl27iYiIuqtboWbLli0ICQmBQqFAdHQ00tLSOtx3+/btEATBYFEoFAb7/Hx767Jx40b9PiEhIW22b9iwoTvlW5R+LnKMZNduIiKie2Z0qNm9ezcSEhKwfv16ZGRkICIiAjNnzkRpacdtQlxdXVFcXKxf8vLyDLbfua24uBhJSUkQBAHz5s0z2O/111832O/55583tnyLpH8FxSkTiIiIus3oULNp0yYsW7YMS5YswbBhw7B161Y4OTkhKSmpw2MEQYBSqdQvvr6+Btvv3KZUKrF3715Mnz4doaGhBvu5uLgY7Ofs7Gxs+RZJ37X7Shm7dhMREXWTUaGmqakJ6enpiI2NvX0CiQSxsbFISUnp8Lja2loEBwcjMDAQc+bMwaVLlzrct6SkBPv378fSpUvbbNuwYQO8vLwwZswYbNy4ERqNpsPzNDY2orq62mCxVKMDPeDWxxHVDRqcK6gydzlERERWyahQU15eDq1W2+ZJi6+vL1QqVbvHhIeHIykpCXv37sXOnTuh0+kwceJEFBYWtrv/jh074OLigkcffdRg/QsvvIBdu3bhyJEjWL58Od588028/PLLHdaamJgINzc3/RIYGGjMrfaqlq7d3gDYC4qIiKi7BFEUxa7uXFRUhICAAJw6dQoxMTH69S+//DKOHTuG1NTUTs/R3NyMoUOHYsGCBXjjjTfabB8yZAgeeOABvPPOO3c9T1JSEpYvX47a2lrI5fI22xsbG9HY2Kj/c3V1NQIDA6FWq+Hq6tppnb3t3+mF+O2eHzAiwBX7np9s7nKIiIgsQnV1Ndzc3Lr0++1gzIm9vb0hlUpRUlJisL6kpARKpbJL53B0dMSYMWOQnZ3dZtuJEyeQmZmJ3bt3d3qe6OhoaDQa5ObmIjw8vM12uVzebtixVFMHt7SruXi9pWu3j4uikyOIiIjoTka9fpLJZIiMjERycrJ+nU6nQ3JyssGTm7vRarW4cOEC/Pz82mz78MMPERkZiYiIiE7Pc+7cOUgkEvj4+HT9BixYPxc5RgS0JNDjV8rNXA0REZH1MepJDQAkJCRg8eLFiIqKwvjx47F582bU1dVhyZIlAID4+HgEBAQgMTERQEs37AkTJmDgwIGoqqrCxo0bkZeXh2eeecbgvNXV1dizZw/eeuutNtdMSUlBamoqpk+fDhcXF6SkpGD16tVYtGgRPDw8unPfFmnaYB9cvF6NY1fK8Fhkf3OXQ0REZFWMDjXz589HWVkZ1q1bB5VKhdGjR+PAgQP6xsP5+fmQSG4/ALpx4waWLVsGlUoFDw8PREZG4tSpUxg2bJjBeXft2gVRFLFgwYI215TL5di1axdeffVVNDY2YsCAAVi9ejUSEhKMLd+iTQvvh3ePZONEVhm0OhFSiWDukoiIiKyGUQ2FrZkxDY3MRaPVYewbB1HdoMH/rZiIyGDbeQpFRETUHcb8fnPuJwviIJXcnrU7k7N2ExERGYOhxsJM5ZQJRERE3cJQY2Gm3erafb5QjfLaxk72JiIiolYMNRbGx1WBYX6tXbv5tIaIiKirGGoskH7Wbk6ZQERE1GUMNRZoWnjLgILHb3XtJiIios4x1FigsUHucFE4oKq+GT8UVpm7HCIiIqvAUGOBWrp2c9ZuIiIiYzDUWKhpg1teQR1jY2EiIqIuYaixUK3j1ZwvrEIFu3YTERF1iqHGQvm6KjBE6QJRBE5kcdZuIiKizjDUWLDWXlBHOWUCERFRpxhqLFjreDXHs8qhY9duIiKiu2KosWCRwR5wkTugsq4J56+rzV0OERGRRWOosWCOUgnuG9jStfvri8VmroaIiMiyMdRYuEfHBgAA/pGSx15QREREd8FQY+EeGOaLkQFuqG/S4r2jOeYuh4iIyGIx1Fg4QRDwXw8OBgD843QeVOoGM1dERERkmRhqrMDUwf0wLsQDjRod3j2SZe5yiIiILBJDjRVoeVoTDgDYfaYABZX1Zq6IiIjI8jDUWIkJoV6YPMgbzVoRbyfzaQ0REdHPMdRYkdanNf+XUYicslozV0NERGRZGGqsyOhAd8QO9YVOBDYf4tMaIiKiOzHUWJmEB1p6Qn35QxEuF1ebuRoiIiLLwVBjZYb5u+IXo/wAAJsOXjFzNURERJaDocYKvRQ7GBIBOPhjCc4VVJm7HCIiIovAUGOFBvr0xaNj+wMA3vo208zVEBERWQaGGiv14oxBcJQKOJFVjtSrFeYuh4iIyOwYaqxUoKcT5o8LBAC89e0ViKJo5oqIiIjMi6HGiq2aPghyBwnScitxPKvc3OUQERGZFUONFVO6KfDkhGAALW1r+LSGiIjsGUONlXtuWhicZFKcL1Tj2x9LzF0OERGR2TDUWDnvvnIsuS8EALDp2yvQ6fi0hoiI7BNDjQ14dnIYXBQOyCypwZfni8xdDhERkVkw1NgANydHPDs5FEDLnFAarc7MFREREfW+boWaLVu2ICQkBAqFAtHR0UhLS+tw3+3bt0MQBINFoVAY7PPUU0+12ScuLs5gn8rKSixcuBCurq5wd3fH0qVLUVvLmapbLZk0AJ7OMlwrr8NnGdfNXQ4REVGvMzrU7N69GwkJCVi/fj0yMjIQERGBmTNnorS0tMNjXF1dUVxcrF/y8vLa7BMXF2ewz6effmqwfeHChbh06RIOHjyIffv24fjx43j22WeNLd9m9ZU7YMXUMADAX5Oz0KjRmrkiIiKi3mV0qNm0aROWLVuGJUuWYNiwYdi6dSucnJyQlJTU4TGCIECpVOoXX1/fNvvI5XKDfTw8PPTbLl++jAMHDuCDDz5AdHQ0Jk2ahHfeeQe7du1CURHbkLR6MiYYPi5yXK+6id1nCsxdDhERUa8yKtQ0NTUhPT0dsbGxt08gkSA2NhYpKSkdHldbW4vg4GAEBgZizpw5uHTpUpt9jh49Ch8fH4SHh2PFihWoqLg99H9KSgrc3d0RFRWlXxcbGwuJRILU1NR2r9nY2Ijq6mqDxdYpHKV4/v6BAIB3DmfjZhOf1hARkf0wKtSUl5dDq9W2edLi6+sLlUrV7jHh4eFISkrC3r17sXPnTuh0OkycOBGFhYX6feLi4vDxxx8jOTkZf/zjH3Hs2DHMmjULWm3Lj7JKpYKPj4/BeR0cHODp6dnhdRMTE+Hm5qZfAgMDjblVqzV/XBAC3PugrKYRO0+3fc1HRERkq0ze+ykmJgbx8fEYPXo0pk6dis8++wz9+vXD+++/r9/niSeewOzZszFy5EjMnTsX+/btw5kzZ3D06NFuX3ft2rVQq9X6paDAPl7HyBwkeDF2EADgvWM5qG3UmLkiIiKi3mFUqPH29oZUKkVJieHItSUlJVAqlV06h6OjI8aMGYPs7OwO9wkNDYW3t7d+H6VS2aYhskajQWVlZYfXlcvlcHV1NVjsxaNjAhDq7YzKuiZ8dPKaucshIiLqFUaFGplMhsjISCQnJ+vX6XQ6JCcnIyYmpkvn0Gq1uHDhAvz8/Drcp7CwEBUVFfp9YmJiUFVVhfT0dP0+hw8fhk6nQ3R0tDG3YBccpBK89MBgAMDfT1yFur7ZzBURERGZntGvnxISErBt2zbs2LEDly9fxooVK1BXV4clS5YAAOLj47F27Vr9/q+//jq+/fZbXL16FRkZGVi0aBHy8vLwzDPPAGhpRPy73/0Op0+fRm5uLpKTkzFnzhwMHDgQM2fOBAAMHToUcXFxWLZsGdLS0vDdd99h1apVeOKJJ+Dv798Tfw825xcj/TBE6YKaBg3+fiLH3OUQERGZnIOxB8yfPx9lZWVYt24dVCoVRo8ejQMHDugbD+fn50MiuZ2Vbty4gWXLlkGlUsHDwwORkZE4deoUhg0bBgCQSqU4f/48duzYgaqqKvj7++PBBx/EG2+8Ablcrj/PJ598glWrVmHGjBmQSCSYN28e3n777Xu9f5slkQhIeGAwnv1HOj76LhdL7hsA777yzg8kIiKyUoIoinYxA2J1dTXc3NygVqvtpn2NKIqYu+U7/FCoxtJJA/DKL4aZuyQiIiKjGPP7zbmfbJggCPivB8MBAP84nYdi9U0zV0RERGQ6DDU2bvIgb4wf4IkmjQ7vHu64xxkREZG1Y6ixcYIg4Le3ntbsPlOA/Ip6M1dERERkGgw1dmD8AE9MGdwPGp2IvyZnmbscIiIik2CosRP/dWvcmv98X4js0hozV0NERNTzGGrsRESgOx4c5gudCPzlEJ/WEBGR7WGosSMJDw6GIAD7zxfjUpHa3OUQERH1KIYaOzJE6YpHRrWMwPyXg1fMXA0REVHPYqixMy/FDoJUIuDQ5VJk5N8wdzlEREQ9hqHGzoT264t5YwMAAJu+5dMaIiKyHQw1duj5+wfBUSrgZHY5UnIqzF0OERFRj2CosUOBnk5YMD4IAPDWt5mwk+m/iIjIxjHU2KmV0wdC7iDB2bwbOHqlzNzlEBER3TOGGjvl66rA4okhAPi0hoiIbANDjR17bmoYnGVSXLxejaOZfFpDRETWjaHGjnk6y/Dr6Ja2NTtScs1bDBER0T1iqLFziyYEQxCAo5llyC2vM3c5RERE3cZQY+eCvZwxbXA/AMDO03lmroaIiKj7GGoI8TEhAIB/nS3AzSateYshIiLqJoYawtTB/RDk6YTqBg32nrtu7nKIiIi6haGGIJEIiI8JBgDsSMlj924iIrJKDDUEAPhVZCAUjhJcLq7G2TxOdElERNaHoYYAAG5Ojpg7umWiy49T2GCYiIisD0MN6T156xXU1xeKUVrdYOZqiIiIjMNQQ3rD/d0QFewBjU7Ep2kF5i6HiIjIKAw1ZKD1ac0nqXlo1urMXA0REVHXMdSQgVkj/ODdV47SmkZ8e6nE3OUQERF1GUMNGZA5SDgfFBERWSWGGmrj1+ODIJUISLtWiZ9U1eYuh4iIqEsYaqgNpZsCccOVANi9m4iIrAdDDbWrtcHwfzKuQ32z2czVEBERdY6hhtoVPcAT4b4uuNmsxf+lF5q7HCIiok4x1FC7BEHQP635x+k86HScD4qIiCwbQw116JdjAuAid8C18jqczC43dzlERER3xVBDHXKWO+CxqP4AgI/ZvZuIiCxct0LNli1bEBISAoVCgejoaKSlpXW47/bt2yEIgsGiUCj025ubm7FmzRqMHDkSzs7O8Pf3R3x8PIqKigzOExIS0uY8GzZs6E75ZIQnJ7S8gkr+qRQFlfVmroaIiKhjRoea3bt3IyEhAevXr0dGRgYiIiIwc+ZMlJaWdniMq6sriouL9Ute3u1uwvX19cjIyMArr7yCjIwMfPbZZ8jMzMTs2bPbnOf11183OM/zzz9vbPlkpNB+fTF5kDdEEdiZyu7dRERkuRyMPWDTpk1YtmwZlixZAgDYunUr9u/fj6SkJPz+979v9xhBEKBUKtvd5ubmhoMHDxqse/fddzF+/Hjk5+cjKChIv97FxaXD85DpxMeE4ERWOXafKcDq2MFQOErNXRIREVEbRj2paWpqQnp6OmJjY2+fQCJBbGwsUlJSOjyutrYWwcHBCAwMxJw5c3Dp0qW7XketVkMQBLi7uxus37BhA7y8vDBmzBhs3LgRGo3GmPKpm+4f4oMA9z6oqm/Glz8UdX4AERGRGRgVasrLy6HVauHr62uw3tfXFyqVqt1jwsPDkZSUhL1792Lnzp3Q6XSYOHEiCgvbH/ukoaEBa9aswYIFC+Dq6qpf/8ILL2DXrl04cuQIli9fjjfffBMvv/xyh7U2NjaiurraYKHukUoELLrVtubjlDyIIrt3ExGR5TH69ZOxYmJiEBMTo//zxIkTMXToULz//vt44403DPZtbm7G448/DlEU8d577xlsS0hI0P/zqFGjIJPJsHz5ciQmJkIul7e5bmJiIl577bUevhv7NX9cIP5y6AouXFfjXEEVxgR5mLskIiIiA0Y9qfH29oZUKkVJSYnB+pKSki63dXF0dMSYMWOQnZ1tsL410OTl5eHgwYMGT2naEx0dDY1Gg9zc3Ha3r127Fmq1Wr8UFBR0qT5qn6ezDLMj/AFwPigiIrJMRoUamUyGyMhIJCcn69fpdDokJycbPI25G61WiwsXLsDPz0+/rjXQZGVl4dChQ/Dy8ur0POfOnYNEIoGPj0+72+VyOVxdXQ0Wujfxt0YY3n++GOW1jWauhoiIyJDRr58SEhKwePFiREVFYfz48di8eTPq6ur0vaHi4+MREBCAxMREAC3dsCdMmICBAweiqqoKGzduRF5eHp555hkALYHmscceQ0ZGBvbt2wetVqtvn+Pp6QmZTIaUlBSkpqZi+vTpcHFxQUpKClavXo1FixbBw4OvQXrLqP7uGB3ojnMFVdh9pgArpw80d0lERER6Roea+fPno6ysDOvWrYNKpcLo0aNx4MABfePh/Px8SCS3HwDduHEDy5Ytg0qlgoeHByIjI3Hq1CkMGzYMAHD9+nV88cUXAIDRo0cbXOvIkSOYNm0a5HI5du3ahVdffRWNjY0YMGAAVq9ebdDOhnpHfEwwzhVUYefpPCyfEgoHKQelJiIiyyCIdtKVpbq6Gm5ublCr1XwVdQ8amrW4b8NhVNQ1YeuiSMSN4LhBRERkOsb8fvM/s8koCkcp5o8LBMD5oIiIyLIw1JDRFk4IhkQATuVUILu0xtzlEBERAWCooW4IcO+DB4a1tKFi924iIrIUDDXULfExIQCA/0svRE1Ds3mLISIiAkMNddPEMC+E9XNGXZMW//n+urnLISIiYqih7hEEQf+0hvNBERGRJWCooW57dGwAnGVSZJfWIiWnwtzlEBGRnWOooW5zUTji0bH9AQA72L2biIjMjKGG7knrfFAHfyzB9aqbZq6GiIjsGUMN3ZNBvi6YGOYFnQj8M5Xdu4mIyHwYauietT6t2ZVWgEaN1szVEBGRvWKooXsWO9QXfm4KVNQ14asLxeYuh4iI7BRDDd0zB6kEC6ODAAA7TvEVFBERmQdDDfWIJ8YHQSaV4FxBFc4XVpm7HCIiskMMNdQjvPvK8dBIJQDOB0VERObBUEM9Jn5iCADgix+KcKOuybzFEBGR3WGooR4zJtAdIwPc0KTRYffZAnOXQ0REdoahhnqMIAh48lb37p2n86DVcT4oIiLqPQw11KNmR/jD3ckRhTdu4shPpSa7jlYn4nJxNT7//jpKqxtMdh0iIrIeDuYugGyLwlGK+VGBeP/4VexIyUXsMN8eOe/NJi1+KKzC2dxKnM27gfS8G6hp0AAA7hvohU+emdAj1yEiIuvFUEM9btGEYPz9xFWcyCrH1bJahPbra/Q5ymsbcTb3hj7EXLyuhuZnr7OcZFLUN2lxKqcCZTWN6Oci76lbICIiK8RQQz0u0NMJ94f7IPmnUvzjdB7WPzL8rvuLooir5XUtASb3Bs7m3cC18ro2+/m6yhEV4omoYA+MC/HEEKULHn3vFM4XqnHwxxL8+tYAgEREZJ8Yasgk4ieGIPmnUvw7vRC/fTAczvLb/6o1aXS4cF2N9LxKnMlteZVU+bMu4IIADPZxQVSIR8sS7In+Hn0gCILBfnEjlDhfqMbXF4sZaoiI7BxDDZnE5IHeGODtjGvldfgkNQ+DfFxw5taTmB8Kq9Co0RnsL3eQICLQXf8UZmyQB9ycHDu9TtxwJf50IBMpORVQ1zd36RgiIrJNDDVkEhKJgEUTgvHGvh/x5lc/tdnu6SxDVPCtpzAhnhjh7waZg/Gd8UL79UW4rwsyS2pw6HIJ5kX274nyiYjICjHUkMk8FtkfW4/loKymEaHezoi89RQmKsQDA7yd27xK6q64EUpkltTg64sqhhoiIjvGUEMm49bHEYf/ayqatSI8nWUmu86skUr8NTkLx7PKUNuoQV85/7UmIrJHHHyPTMpF4WjSQAMA4b4uCPFyQpNGh6OZphvwj4iILBtDDVk9QRAQN8IPAPD1RZWZqyEiInNhqCGbMGuEEgBw5KdSNDRrzVwNERGZA0MN2YRR/d3g76ZAfZMWJ7LKzV0OERGZAUMN2QRBEDDz1tOary8Wm7kaIiIyB4YashmzbrWrOfRjCZp+NrgfERHZPoYashmRwR7w7itHdYMGKVcrzF2O2Z3NrUR2aa25yyAi6jUMNWQzpBIBDw73BQAcsPNeUNfK6zD/76fx622nof3Z7OZERLaKoYZsSmsvqIM/quz6x/zwT6XQ6kSU1jTih8Iqc5dDRNQruhVqtmzZgpCQECgUCkRHRyMtLa3Dfbdv3w5BEAwWhUJhsI8oili3bh38/PzQp08fxMbGIisry2CfyspKLFy4EK6urnB3d8fSpUtRW8tH62RoQqgX3Po4ory2CWdyK81djtkcu1J2+58zy+6yJxGR7TA61OzevRsJCQlYv349MjIyEBERgZkzZ6K0tOORXF1dXVFcXKxf8vLyDLb/6U9/wttvv42tW7ciNTUVzs7OmDlzJhoaGvT7LFy4EJcuXcLBgwexb98+HD9+HM8++6yx5ZONc5RKEDvUvl9BNTRrkXpHm6I7Aw4RkS0zOtRs2rQJy5Ytw5IlSzBs2DBs3boVTk5OSEpK6vAYQRCgVCr1i6+vr36bKIrYvHkz/ud//gdz5szBqFGj8PHHH6OoqAiff/45AODy5cs4cOAAPvjgA0RHR2PSpEl45513sGvXLhQVFRl/12TTWl9BfXNJBZ0dvoJKvVaJRo0Obn0cAQA/FFbhRl2TmasiIjI9o0JNU1MT0tPTERsbe/sEEgliY2ORkpLS4XG1tbUIDg5GYGAg5syZg0uXLum3Xbt2DSqVyuCcbm5uiI6O1p8zJSUF7u7uiIqK0u8TGxsLiUSC1NTUdq/Z2NiI6upqg4Xsw6RB3nCWSVGsbrDL9iStr5tmjVBiiNIFogiczOaAhERk+4wKNeXl5dBqtQZPWgDA19cXKlX7j/rDw8ORlJSEvXv3YufOndDpdJg4cSIKCwsBQH/c3c6pUqng4+NjsN3BwQGenp4dXjcxMRFubm76JTAw0JhbJSumcJTifjt+BXU8qyXUTBncD1MG9wPAV1BEZB9M3vspJiYG8fHxGD16NKZOnYrPPvsM/fr1w/vvv2/S665duxZqtVq/FBQUmPR6ZFnihre8gjpwSQVRtJ9XUNerbiK7tBZSiYD7Bnpj6q1Qc/xKmV39PRCRfTIq1Hh7e0MqlaKkpMRgfUlJCZRKZZfO4ejoiDFjxiA7OxsA9Mfd7ZxKpbJNQ2SNRoPKysoOryuXy+Hq6mqwkP2YFt4PcgcJ8irqcbm4xtzl9Jrjt57IjA50h1sfR0SFeKCPoxSlNY34SWU/fw9EZJ+MCjUymQyRkZFITk7Wr9PpdEhOTkZMTEyXzqHVanHhwgX4+bUMaT9gwAAolUqDc1ZXVyM1NVV/zpiYGFRVVSE9PV2/z+HDh6HT6RAdHW3MLZCdcJY76J9SHLCjuaBa29O03rvcQYqJYV4t2/gKiohsnNGvnxISErBt2zbs2LEDly9fxooVK1BXV4clS5YAAOLj47F27Vr9/q+//jq+/fZbXL16FRkZGVi0aBHy8vLwzDPPAGjpGfXSSy/hD3/4A7744gtcuHAB8fHx8Pf3x9y5cwEAQ4cORVxcHJYtW4a0tDR89913WLVqFZ544gn4+/v3wF8D2aK4EbdfQdmDZq0O391qENzalubOf+Z4NURk6xyMPWD+/PkoKyvDunXroFKpMHr0aBw4cEDf0Dc/Px8Sye2sdOPGDSxbtgwqlQoeHh6IjIzEqVOnMGzYMP0+L7/8Murq6vDss8+iqqoKkyZNwoEDBwwG6fvkk0+watUqzJgxAxKJBPPmzcPbb799L/dONm7GUF84SgVcKalFTlktwvr1NXdJJnWuoAo1jRp4ODliZICbfn3rU5uzeZWoa9TAWW70156IyCoIop20HqyuroabmxvUajXb19iRxUlpOHalDL+bGY6V0weauxyT+vM3mXj3SDZmR/jj7QVjDLZN3XgEeRX1+CA+CrHDfDs4AxGR5THm95tzP5FNax2I72s7aFdzZ1fun5syiF27icj2MdSQTXtgmC8kAnDxejUKKuvNXY7JVNQ24sJ1NQBgyiDvNtuncrwaIrIDDDVk07z6yjF+gCeAlmkTbNXJ7HKIIjDUzxU+roo222PCvOAoFZBfWY/c8jozVEhEZHoMNWTzZo1oGT7gaxseXfjnXbl/zlnugKjglnDHpzVEZKsYasjmzbw1unB63g2UVjd0srf10elEfXuajkINAEwN5ysoIrJtDDVk85RuCowJcgdgm6+gfiyuRnltE5xlUkQGe3S4X2vgScmpQKNG21vlERH1GoYasgu3e0HZXqhpffISE+YNmUPHX+khShf4uMhxs1mLs7k3eqs8IqJew1BDdiFueEu7mtRrlaisazJzNT2rdb6nqYPb9nq6kyAInLWbiGwaQw3ZhSAvJwzzc4VWJ+LQjyWdH2AlahqakZ7X8tRl6mCfTvefyikTiMiGMdSQ3bDFgfhO5VRAoxMR4uWEIC+nTvefNNAbEgHILKmBSm17jaaJyL4x1JDdmDWyJdSczC5HdUOzmavpGbdfPXXc6+lOHs4yjOrvbnAsEZGtYKghuzHQxwVh/ZzRrBVx5KdSc5dzz0RR1LeNae2u3RUcXZiIbBVDDdkV/UB8F6y/F9S18joU3rgJmVSCCaFeXT6uNQCdzC6HRqszVXlERL2OoYbsStytdjVHr5Sivklj5mruTeuTlnEDPOAkc+jycRH93eHWxxHqm834oVBtqvKIiHodQw3ZleH+rujv0QcNzTqr7wHU2iamdQburpJKBEy6NeklX0ERkS1hqCG7IgiCvhfUASseXbihWYuUqxUAjGtP04rtaojIFjHUkN2Ju9Wu5vDlUqudLuBMbiUamnXwdZUj3NfF6ONbQ835wircsLHBCInIfjHUkN0ZE+gOX1c5aho1+C673NzldMudr54EQTD6eF9XBYYoXSCKwAkr/TsgIvo5hhqyOxKJoJ+5+4CVzgXVna7cP8fRhYnI1jDUkF1q7QV18McSq+vWXKy+iSsltZAILSMEd1drqDmeVQZRFHuqPCIis2GoIbs0PsQTns4y3KhvRuq1SnOXY5TWV08Rge5wd5J1+zyRIR7o4yhFWU0jLhfX9FR5RERmw1BDdslBKsEDQ30BWN9cUMe62ZX75+QOUkwM8zI4JxGRNWOoIbsVd2suqG8ulUCns47XLxqtDiezWhr23kt7mlat5+A8UERkCxhqyG7dF+YNF4UDymoakZF/w9zldMkPhVWobtDArY8jIm5NTHkvWtvVnM2rRG2jdY+wTETEUEN2S+YgQaz+FZR19II6dqXlKc2kQd6QSozvyv1zwV7OCPZyQrNWREpOxT2fj4jInBhqyK7d2bXbGnoA6btyD773V0+tbo8ubP0zlxORfWOoIbs2dXA/9HGU4nrVTVy8Xm3ucu7qRl0TzhdWAbj3RsJ3unPKBGsIdkREHWGoIbvWRybF9CEtP+qW3gvqRHY5RBEYonSB0k3RY+edEOoFR6mAgsqbyK2o77HzEhH1NoYasnvW8gqqdeTfKT346gkAnOUOGBfieesafAVFRNaLoYbs3v1DfCCTSnC1vA5ZpbXmLqddoijieFbPt6dpdXt0Yc4DRUTWi6GG7J6LwhGTB7VMN/D1BcvsBXW5uAZlNY3o4yhFVIhHj5+/9elPSk4FGpqtc+ZyIiKGGiLcngvKUtvVtD6liQnzgtxB2uPnH6J0gY+LHDebtTibax1j9hAR/RxDDRGA2KG+kEoE/KSqQW55nbnLaaO1PY0pXj0BgCAI7NpNRFaPoYYIgIezDDGhLfMgHbhkWa+g6ho1OJvXMulmTzcSvtPtKRPYroaIrBNDDdEtt19BWVaoScmpQLNWRJCnE0K8nEx2nUkDvSERgMySGhSrb5rsOkREptKtULNlyxaEhIRAoVAgOjoaaWlpXTpu165dEAQBc+fONVgvCEK7y8aNG/X7hISEtNm+YcOG7pRP1K4Hh/tCEIAfCqpQVGU5P+p3jiIsCPc+NUJH3J1kiAh0B8AJLonIOhkdanbv3o2EhASsX78eGRkZiIiIwMyZM1Faevf38Lm5ufjtb3+LyZMnt9lWXFxssCQlJUEQBMybN89gv9dff91gv+eff97Y8ok65OOiQFRwS8+ibyzoFVRrI2FTvnpqpe/azVdQRGSFjA41mzZtwrJly7BkyRIMGzYMW7duhZOTE5KSkjo8RqvVYuHChXjttdcQGhraZrtSqTRY9u7di+nTp7fZ18XFxWA/Z2dnY8snuqu4EX4ALOcVVG55HfIq6uEoFRAT5mXy67UGpxNZZdBodSa/HhFRTzIq1DQ1NSE9PR2xsbG3TyCRIDY2FikpKR0e9/rrr8PHxwdLly7t9BolJSXYv39/u/tu2LABXl5eGDNmDDZu3AiNRtPheRobG1FdXW2wEHWmtV3NmdxKlNU0mrma209pIoM90FfuYPLrRfR3h1sfR1Q3aPDDrXmmiIishVGhpry8HFqtFr6+vgbrfX19oVK1/1+2J0+exIcffoht27Z16Ro7duyAi4sLHn30UYP1L7zwAnbt2oUjR45g+fLlePPNN/Hyyy93eJ7ExES4ubnpl8DAwC5dn+xbgHsfjOrvBlEEDv5YYu5y7ujK7dMr15NKBP1AhK3XJiKyFibt/VRTU4Mnn3wS27Ztg7e3d5eOSUpKwsKFC6FQGE7Yl5CQgGnTpmHUqFF47rnn8NZbb+Gdd95BY2P7/zW9du1aqNVq/VJQUHDP90P2wVIG4mvUaHEqpwIAMGVw174/PaH1FdQxTplARFbGqOfZ3t7ekEqlKCkx/C/YkpISKJXKNvvn5OQgNzcXjzzyiH6dTtfynt7BwQGZmZkICwvTbztx4gQyMzOxe/fuTmuJjo6GRqNBbm4uwsPD22yXy+WQy+VdvjeiVnHDlfjTgUyk5FRAXd8MNydHs9SRnnsDN5u16OcixzA/1167bmtj4fOFVaisa4Kns6zXrk1EdC+MelIjk8kQGRmJ5ORk/TqdTofk5GTExMS02X/IkCG4cOECzp07p19mz56N6dOn49y5c21eCX344YeIjIxEREREp7WcO3cOEokEPj6981ie7Edov74I93WBRifi0GXzvYJq7co9ZZBpu3L/nK+rAkOULhDFlgbDRETWwuiWhwkJCVi8eDGioqIwfvx4bN68GXV1dViyZAkAID4+HgEBAUhMTIRCocCIESMMjnd3dweANuurq6uxZ88evPXWW22umZKSgtTUVEyfPh0uLi5ISUnB6tWrsWjRInh49PzkfkRxI5TILKnB1xdVmBfZ3yw16ENNL756ajU1vB9+UtXg+JVyzBkd0OvXJyLqDqNDzfz581FWVoZ169ZBpVJh9OjROHDggL7xcH5+PiQS45vq7Nq1C6IoYsGCBW22yeVy7Nq1C6+++ioaGxsxYMAArF69GgkJCUZfh6grZo1U4q/JWTieVYbaRk2v9Dy6U0l1A35S1UAQgMmDTD8+zc9NHdQP7x+7imNXyqDTiZBIeu9JERFRdwmiKIrmLqI3VFdXw83NDWq1Gq6uvdc+gayTKIqY/uejyK2ox//+cgQWRgf36vX/dbYAL//7PCL6u2Hvqkm9em2gpZHymNcPor5Ji/0vTMJwf7der4GICDDu95tzPxG1QxAE/Cqqpc3Xa1/8iNNXK3r1+sevmHZW7s7IHaSYeGuwP44uTETWgqGGqAPPTQ3Dg8N80aTVYdnHZ/GTqncGcNTqRJy41Z26N6ZG6Ii+a/eVu0+BQkRkKRhqiDoglQh4e8EYRAV7oKZBg8VJabjeCxNdni+sgvpmM1wUDhh9a4JJc2h9SnQ29wZqGzsevZuIyFIw1BDdhcJRig8WR2GQT1+UVDci/sNU3KhrMuk1W3s9TR7kDQep+b6iwV7OCPFygkYn4lQ2X0ERkeVjqCHqhLuTDDueHg+lqwI5ZXVYuuMMbjZpTXa943eMT2Nura+gjnO8GiKyAgw1RF3g794HHy8dD1eFAzLyq/D8p9+bZBbrqvomnCuoAmDe9jStWl9BHc0sg510lCQiK8ZQQ9RFg31d8MHicZA5SHDocgle2Xuxx3/oT2aXQycCg3z6wt+9T4+euzsmhHpBJpWg8MZNXCuvM3c5RER3xVBDZITxAzzx9hNjIBGAT9MKsPlQVo+e39xduX/OWe6AcQNaRu1urY2IyFIx1BAZKW6EEq/PaZnm46/JWfgkNa9HziuKor6R8NRwywg1wO22PccYaojIwjHUEHXDognBeOH+gQCAVz6/iG8uqe75nFdKalFS3QiFowTjQjzv+Xw9pTVgpVytQEOz6RpIExHdK4Yaom5a/cBgzI8KhE4EXvj0e5zJrbyn87UOcjch1AsKR2lPlNgjwn1d4OsqR0Oz7p7vkYjIlBhqiLpJEAT87y9HYMYQHzRqdFi6/QyulNR0+3zHLKgr950EQdDXxHY1RGTJGGqI7oGDVIJ3fz0WY4LcUX1r1OFitfGjDtc3aXDm2g0AltWeplVrTWxXQ0SWjKGG6B71kUmRtHgcQvs5o1jdgMVJaVDXNxt1jtNXK9Ck1SHAvQ9CvZ1NVGn3TRroDYnQ0u6nqBemiiAi6g6GGqIe4OEsw8dPj4ePixxXSmqx7OOzRjWqbZ0Je2p4PwiCYKoyu83dSaafh+oERxcmIgvFUEPUQ/p7OGHH0+PhIndAWm4lXtz1PbS6rg3Od8zCxqdpz+1ZuxlqiMgyMdQQ9aChfq74e3wUZFIJvrlUgnVdGHU4v6Ie18rr4CARMDHMq5cqNV5r4DqRVW6SKSKIiO4VQw1RD4sJ88Jf5o+GIACfpObj3cPZd93/2K3XOWODPeCicOyNErtlVH93uDs5oqZBgx8Kq8xdDhFRGww1RCbw8Cg/rP/FMADAWwevYFdafof7Hsu0/FdPACCVCJg00BvA7ZqJiCwJQw2RiTx13wD8ZloYAOD//ecCDv1Y0mafJo0OKTm3GglbeKgBbtfIdjVEZIkYaohM6HczwzFvbH/oRGDVpxlIz7thsD097wbqmrTw7ivDMD9XM1XZda2h5vx1NSrrmsxcDRGRIYYaIhMSBAEb5o3EtPB+aGjWYemOM8gurdVvP36rPc3kQf0gkVheV+6f83FVYIjSBaLIrt1EZHkYaohMzFEqwd8WjkVEfzdU1TdjcVIaSqobAFhPe5o7cXRhIrJUDDVEvcBJ5oCkp8ZhgLczrlfdxOKkNOSU1eLH4moAwKRB3mausOtaA9jxK+XQdXEcHiKi3sBQQ9RLvPrK8fHT49HPRY6fVDV4fGsKAGBkgBu8+8rNXF3XRQV7wkkmRXltIy6rqs1dDhGRHkMNUS8K9HTCR0+NQ1+5AypuNbSdMth6ntIAgMxBoh8k8GA7PbqIiMyFoYaol40IcMP7T0bCUdrSMHh6uI+ZKzJe7FBfAMDbyVnYeTrPzNUQEbVwMHcBRPbovoHe+HTZBFwrr0NUiKe5yzHaY5H9kZF/A/86W4j/+fwiCm/cxMszw62iBxcR2S5B7GxiGhtRXV0NNzc3qNVquLpa/nggRJZOFEW8czgbmw5eAQA8EuGPP/9qFOQOUjNXRkS2xJjfb75+IqJuEQQBL8wYhLd+FQEHiYAvfyjCkx+koaqeg/IRkXkw1BDRPZkX2R87nh4PF7kD0nIr8eh7p1BQWW/usojIDjHUENE9u2+gN/69YiL83RS4WlaHX/7tO/xQUGXusojIzjDUEFGPCFe64D8r78MwP1eU1zbhib+fZpdvIupVDDVE1GN8XRX413MxmDq4H242a7H8H2fxcUquucsiIjvBUENEPaqv3AEfLI7CE+MCoROBdXsv4c2vLnNKBSIyuW6Fmi1btiAkJAQKhQLR0dFIS0vr0nG7du2CIAiYO3euwfqnnnoKgiAYLHFxcQb7VFZWYuHChXB1dYW7uzuWLl2K2tpaEJHlcZRKkPjoSPxuZjgA4O/Hr+L5T79HQ7PWzJURkS0zOtTs3r0bCQkJWL9+PTIyMhAREYGZM2eitLT0rsfl5ubit7/9LSZPntzu9ri4OBQXF+uXTz/91GD7woULcenSJRw8eBD79u3D8ePH8eyzzxpbPhH1EkEQsHL6QGyePxqOUgH7LxRj0QepuFHHLt9EZBpGD74XHR2NcePG4d133wUA6HQ6BAYG4vnnn8fvf//7do/RarWYMmUKnn76aZw4cQJVVVX4/PPP9dufeuqpNuvudPnyZQwbNgxnzpxBVFQUAODAgQN46KGHUFhYCH9//07r5uB7ROZzKqccy/+RjpoGDUK9nfHRknEI9nI2d1lEZAVMNvheU1MT0tPTERsbe/sEEgliY2ORkpLS4XGvv/46fHx8sHTp0g73OXr0KHx8fBAeHo4VK1agoqJCvy0lJQXu7u76QAMAsbGxkEgkSE1Nbfd8jY2NqK6uNliIyDwmhnnj/1ZMRIB7H1wtr8OjfzuF7/NvmLssIrIxRoWa8vJyaLVa+Pr6Gqz39fWFSqVq95iTJ0/iww8/xLZt2zo8b1xcHD7++GMkJyfjj3/8I44dO4ZZs2ZBq215/65SqeDjYzjpn4ODAzw9PTu8bmJiItzc3PRLYGCgMbdKRD1ssK8L/vObiRgR4IqKuiYs2HYa315q//tLRNQdJu39VFNTgyeffBLbtm2Dt7d3h/s98cQTmD17NkaOHIm5c+di3759OHPmDI4ePdrta69duxZqtVq/FBQUdPtcRNQzfFwV2P1sDKaH90NDsw7Ld6Zj+3fXzF0WEdkIo2bp9vb2hlQqRUmJ4YBaJSUlUCqVbfbPyclBbm4uHnnkEf06nU7XcmEHB2RmZiIsLKzNcaGhofD29kZ2djZmzJgBpVLZpiGyRqNBZWVlu9cFALlcDrlcbsztEVEvcJY7YFt8FNZ9cQn/TM3Hq1/+iMIbN/H/HhrKWb6J6J4Y9aRGJpMhMjISycnJ+nU6nQ7JycmIiYlps/+QIUNw4cIFnDt3Tr/Mnj0b06dPx7lz5zp8JVRYWIiKigr4+fkBAGJiYlBVVYX09HT9PocPH4ZOp0N0dLQxt0BEFsBBKsH/zh2BNXFDAAAfnLyGlf/MYJdvIronRj2pAYCEhAQsXrwYUVFRGD9+PDZv3oy6ujosWbIEABAfH4+AgAAkJiZCoVBgxIgRBse7u7sDgH59bW0tXnvtNcybNw9KpRI5OTl4+eWXMXDgQMycORMAMHToUMTFxWHZsmXYunUrmpubsWrVKjzxxBNd6vlERJZHEASsmBYGf3cFfrfnPL6+qEJJ9Wl8sHgcPJ1l5i6PiKyQ0aFm/vz5KCsrw7p166BSqTB69GgcOHBA33g4Pz8fEknXHwBJpVKcP38eO3bsQFVVFfz9/fHggw/ijTfeMHh99Mknn2DVqlWYMWMGJBIJ5s2bh7ffftvY8onIwswZHQClqwLLPj6LjPwqPPq377B9yXiEeLPLNxEZx+hxaqwVx6khsmzZpTV46qMzKLxxE57OMmyLj0JksIe5yyIiMzPZODVERKYy0McFn/1mIkYGuKGyrgm/3nYa//P5Bfyk4hhTRNQ1fFJDRBalvkmDFz79Hocu3+7xGBnsgUUTgjBrhB8UjlIzVkdEvc2Y32+GGiKyOKIoIiWnAjtT8/DtpRJobs3w7eHkiF9FBeLX44PY5obITjDUtIOhhsg6lVY3YPeZAnyalo8idYN+/eRB3lgYHYzYoT5wkPJNOpGtYqhpB0MNkXXTaHU4klmGT1LzcOxKGVr/n8vXVY4nxgVhwfggKN0U5i2SiHocQ007GGqIbEd+RT3+mZaPPWcLUFHXBACQSgTEDvXBwuhgTBrozdGJiWwEQ007GGqIbE+jRosDF1X4JDUfadcq9etDvJzw6+gg/CoyEB4cyI/IqjHUtIOhhsi2XSmpwSen8/BZxnXUNGoAADIHCR4e6YdFE4IwNsgDgsCnN0TWhqGmHQw1RPahvkmDL84VYWdqHi5evz3GzRClCxZOCMYvxwSgr9zowdSJyEwYatrBUENkX0RRxPlCNXaezsOX54vQ0KwDADjLpJgzJgDxMcEYouT/FxBZOoaadjDUENkvdX0z/i+jEJ+k5iGnrE6/Pm64EqsfGIxwpYsZqyOiu2GoaQdDDRGJoojTVyvxj9O5+PqiCqIICALwi1H+eCl2EML69TV3iWTBGpq1qL7ZjH4ucrbP6kUMNe1gqCGiO10pqcHmQ1fw1QUVAEAiAHPHBODFGYMQ7MXRiu1ZXaMGOWW1yCqpRXbr/5bWIL+yHjqxZeDH12YPRyhDcK9gqGkHQw0RtedSkRp/OXhFP9eUg0TAY5H98fyMQQhw72Pm6siU1DebkV3aEliySmqRVVqL7NJaXK+62emxMqkEz04JxcrpA9FHxvnITImhph0MNUR0N+cKqrDp4BUcv1IGoOVH64nxgVg5fSB8XTlSsTWrqG1EVmlLaMkprUXWrRBTWtPY4THefWUY6NMXg3xcMMi3Lwb6tCz1jVqs/+ISjt369yTAvQ/WPzIMDwzz5SspE2GoaQdDDRF1xdncSrz17RWkXK0AAMgdJFg0IRjPTQ1DPxe5maujznyffwPnCqr0T12yS2tReWvU6fb4uSn0gUUfYPr1veugjaIo4ptLJXhj34/6pzrTw/vh1dnD+erSBBhq2sFQQ0TGOJVTjk3fXsHZvBsAgD6OUsRPDMZzU8I4SrEF0mh1+MP+y9h+KrfNNkEA+nv0aQktPn0R5tMXg24FGReFY7evWd+kwbuHs7HtxFU0a0XIHCRYMTUMK6aFQeHIV1I9haGmHQw1RGQsURRxPKscm77NxA+FagAt49wsnTQASyeHwq1P938Qqeeo65ux6tMMnMgqBwDMGOKDIX4uGOTjgoE+fRHWr69J273klNVi/d5LOJndcv0gTye8OnsY7h/ia7Jr2hOGmnYw1BBRd4miiOTLpdh08Ap+LG4ZpdhV4YBlk0OxZNIAjlBsRlfLavHMjrO4Wl6HPo5S/GV+BOJG+PV6HaIo4qsLKryx70eoqhsAAA8M88W6XwxDoKdTr9djSxhq2sFQQ0T3SqcT8c0lFf5y6AqulNQCADycHLF8ahjiY4LhJGO46U0ns8rxm0/SUd2ggb+bAtsWR2G4v5tZa6pr1ODt5Cx8ePIaNDoRCkcJVk0fiGVTQiF34Cup7mCoaQdDDRH1FK1OxL7zRfjroSxcLW8Zodi7rwzPTQ3DognBPdaeQqcTUd+sRV2jBrWNGtQ3alHbqEGjRgt/9z4I8nSy27Yb/0jJxatf/gitTsSYIHe8/2QkfFwsp5falZIarNt7EaevtsweP8DbGa/NHo4pg/uZuTLrw1DTDoYaIuppGq0On58rwl+Tr6CgsqUXjK+rHCunD8TEMC/UNd4RSJpaAkld69LU+bb6Ju1dry8ILV2KB3g7I9TbGQO8nTGgX1+EejvD370PpBLb62LcrNXhtS8vYefpfADAL8cEIPHRkRYZ7kRRxBc/FOEP+y+j7Fb38VkjlHjlF8PgzzGQuoyhph0MNURkKs1aHf6dXoh3krNQpG7o8fNLBMBZ7oC+cgc4yaRwlEpw/cZN1DRqOjxG5iBBiJdTS9Dxbgk6A/q1BB8vZ5lVjqlSVd+Elf/MwHfZFRAE4Hczw7FiapjF30tNQzM2H8rC9lO50OpE9HGU4oUZg7B00gDIHCTmLq9HiKKIY1fK4OuqwFC/nv2NZahpB0MNEZlao0aL3WcKsO3EVajrm9FX7gBnuQOc5A7oK5fCWXYrmMilLSFF1rLd+daf7wwurcc6yxygcJS0+eEWRREVdU24Vl6Ha2V1uFpeh2vltbhWXofc8no0aXUd1umicLj9ZMe7L0L7tf6zM5wttNFzdmktntlxBrkV9XCSSbF5/mg8OFxp7rKMcrm4Guv2XsSZ3JZhAsL6OeONOSMwcaC3mSvrPp1OxLc/qrDlSA4uXFcjdqgvPlgc1aPXYKhpB0MNEdkLrU5EUdXNlqBT1hJ0WkJPHa5X3cTd/l/f11WOIUpXLJoQjBlDfCCxgFdYx6+UYeU/M1DToEGAex98sDiqx58G9BZRFPFZxnUkfn0Z5bUtgwI+EuGP/35oKJRultMmqDMarQ77zhdjy5FsZJW2NJrv4yjFoglBWDtraI/+e8NQ0w6GGiKilpmm8yvrcbWs9lboaQk718rrUPGzkXcH+/bFimlheGSUPxykvf+aRBRFbD+Vizf2/QidCEQFe2Drk5Hw7mv9IzurbzZj07eZ+MfpPOjElvGPVj8wGIsnhsDRDH/XXdWo0eKzjOt472gO8ivrAbQ8+XtqYgiW3DcAniYYmJKhph0MNUREd6eub8bV8lp8c6kEO0/nofZWm53+Hn2wfEoofhUV2GsNcpu1OqzbewmfprU0CJ43tj/efHSEzXWLvnhdjVf2XsT3+VUAgNB+zvhVZCAeHumHIC/LGd/mZpMWu87k4+/Hr6L4VrsxT2cZlk4agCdjguF6DyMzd4ahph0MNUREXae+2Yydp/OQdPKa/gmOd18Zltxn+h+xG3VNWPFJOk5frYQgAGtnDcGyyaEW3yC4u3Q6Ef9OL8SGAz8ZzFMV0d8ND4/yw8Oj/M02Y3xNQzP+cToPH564/e+Br6scyyaH4tfRQb0yNhNDTTsYaoiIjNfQrMW/zhbg/WNX9ZM3usgdsCgmGE/fN6DHJ/nMKqnB0h1nkV9ZD2eZFG8vGIMZQ+1jugH1zWbsP1+M/ReKkJJTAd0dv85jgtzxi1H+eGikEn5upg84N+qa8NGpXGz/7hqqG24/sVsxLQyPRfbv1SdmDDXtYKghIuq+Zq0OX/5QhPeO5ugbhsodJHg8KhDPTgntkakAjmSW4oV/fo+aRg36e/TBh4vHIVzpcs/ntUZlNY04cEmFfT8UIS230qBx97gQDzw80g8PjfSDj2vPNi4urWnAByeuYefpPP04SWH9nPGbaQMxe7S/Wdr7MNS0g6GGiOje6XQikn8qxd+OZuvbgUglAmZH+OO5qWHdCiGiKOLDk9fw5leXoROB8SGeeG/RWHjZQIPgnlBa3YCvLhRj/4VifXdwoGXwxfEhnvhFhD9mjVDeUwPq61U38f6xHOw6U4AmTctwAEP9XPH8/QMxc7jSrAM5MtS0g6GGiKjniKKI01cr8bej2frZsQEgdqgPVkwbiMhgjy6dp0mjw7q9F7HrTAEAYH5UIN6YO8JmBqXracXqm/jqggr7zhfpQyXQMkBjTJgXHh7pj7gRyi73QrpWXof3jmbjs4zr0Nx63zU2yB2r7h+I6eE+FtGOiaGmHQw1RESmcaFQja3HcvDVxWL9a5LoAZ74zfSBmDLIu8Mfxsq6Jjy3Mx1p1yohEYD/99BQLJ00wCJ+SK1B4Y36lic454vxQ6Fav14qEXDfQG/8YqQfHhzuC3entgHnJ1U1thzJwf7zRfq2OxPDvLDq/oGICfWyqM+AoaYdDDVERKZ1tawW7x+7is++L0SztuWnZbi/K1ZMC8OsEX4GrzCulNRg6Y4zKKi8CRe5A97+9RhMD/cxV+lWL7+iHvsvFGPf+SJcKqrWr3eUCpg00BsPj/LHg8N9cbWsDu8ezsahyyX6fWYM8cHK+wdibFDXnq71NoaadjDUEBH1jmL1TXxw4ho+TcvXNzYd4O2M5VNC8cuxAfguuxwvfHoOtY0aBHk64cPFURjka58Ngk3hWnkd9p8vwr7zxfhJVaNf7yAR9K+YBAF4aIQffjM9DMP93cxVapcY8/vdrZeWW7ZsQUhICBQKBaKjo5GWltal43bt2gVBEDB37lz9uubmZqxZswYjR46Es7Mz/P39ER8fj6KiIoNjQ0JCIAiCwbJhw4bulE9ERCbk59YHr/xiGL5bcz9eih0EdydHXCuvw+8/u4D7NhzB0h1nUduowYRQT+xdeR8DTQ8b4O2MVfcPwoGXpuBQwhSsjh2MQT59odGJkEoEPDo2AAdXT8WWhWMtPtAYy+gnNbt370Z8fDy2bt2K6OhobN68GXv27EFmZiZ8fDp+dJibm4tJkyYhNDQUnp6e+PzzzwEAarUajz32GJYtW4aIiAjcuHEDL774IrRaLc6ePas/PiQkBEuXLsWyZcv061xcXODs7NyluvmkhojIPOoaNfg0LR8fnLgGVXXLaLQLxgfhtdnD2SC4F+WW18FJJu3xbuCmZtLXT9HR0Rg3bhzeffddAIBOp0NgYCCef/55/P73v2/3GK1WiylTpuDpp5/GiRMnUFVVpQ817Tlz5gzGjx+PvLw8BAUFAWgJNS+99BJeeuklY8rVY6ghIjKvRo0WX19QwVEqwUMjlRbVGJUsl8lePzU1NSE9PR2xsbG3TyCRIDY2FikpKR0e9/rrr8PHxwdLly7t0nXUajUEQYC7u7vB+g0bNsDLywtjxozBxo0bodFoOjxHY2MjqqurDRYiIjIfuYMUc8cE4OFRfgw0ZBJGTdpQXl4OrVYLX1/DIat9fX3x008/tXvMyZMn8eGHH+LcuXNdukZDQwPWrFmDBQsWGCSyF154AWPHjoWnpydOnTqFtWvXori4GJs2bWr3PImJiXjttde6dmNERERk9Uw6E1VNTQ2efPJJbNu2Dd7e3p3u39zcjMcffxyiKOK9994z2JaQkKD/51GjRkEmk2H58uVITEyEXN52FMW1a9caHFNdXY3AwMB7uBsiIiKyZEaFGm9vb0ilUpSUlBisLykpgVKpbLN/Tk4OcnNz8cgjj+jX6XQtwy87ODggMzMTYWFhAG4Hmry8PBw+fLjT92bR0dHQaDTIzc1FeHh4m+1yubzdsENERES2yag2NTKZDJGRkUhOTtav0+l0SE5ORkxMTJv9hwwZggsXLuDcuXP6Zfbs2Zg+fTrOnTunf3LSGmiysrJw6NAheHl5dVrLuXPnIJFI7trjioiIiOyH0a+fEhISsHjxYkRFRWH8+PHYvHkz6urqsGTJEgBAfHw8AgICkJiYCIVCgREjRhgc39r4t3V9c3MzHnvsMWRkZGDfvn3QarVQqVQAAE9PT8hkMqSkpCA1NRXTp0+Hi4sLUlJSsHr1aixatAgeHpY5AiIRERH1LqNDzfz581FWVoZ169ZBpVJh9OjROHDggL7xcH5+PiSSrj8Aun79Or744gsAwOjRow22HTlyBNOmTYNcLseuXbvw6quvorGxEQMGDMDq1asN2swQERGRfeM0CURERGSxTD5NAhEREZGlYaghIiIim8BQQ0RERDaBoYaIiIhsAkMNERER2QSGGiIiIrIJJp37yZK09lznbN1ERETWo/V3uysj0NhNqKmpqQEATmpJRERkhWpqauDm5nbXfexm8D2dToeioiK4uLhAEARzl2MyrbORFxQU2MUgg/Z0v7xX22VP98t7tV2mul9RFFFTUwN/f/9OZyywmyc1EokE/fv3N3cZvcbV1dUuvkSt7Ol+ea+2y57ul/dqu0xxv509oWnFhsJERERkExhqiIiIyCYw1NgYuVyO9evXQy6Xm7uUXmFP98t7tV32dL+8V9tlCfdrNw2FiYiIyLbxSQ0RERHZBIYaIiIisgkMNURERGQTGGqIiIjIJjDUWJHExESMGzcOLi4u8PHxwdy5c5GZmXnXY7Zv3w5BEAwWhULRSxXfm1dffbVN7UOGDLnrMXv27MGQIUOgUCgwcuRIfPXVV71U7b0JCQlpc6+CIGDlypXt7m9tn+vx48fxyCOPwN/fH4Ig4PPPPzfYLooi1q1bBz8/P/Tp0wexsbHIysrq9LxbtmxBSEgIFAoFoqOjkZaWZqI76Lq73WtzczPWrFmDkSNHwtnZGf7+/oiPj0dRUdFdz9md70Jv6Oxzfeqpp9rUHRcX1+l5LfFzBTq/3/a+w4IgYOPGjR2e0xI/26781jQ0NGDlypXw8vJC3759MW/ePJSUlNz1vN39nhuDocaKHDt2DCtXrsTp06dx8OBBNDc348EHH0RdXd1dj3N1dUVxcbF+ycvL66WK793w4cMNaj958mSH+546dQoLFizA0qVL8f3332Pu3LmYO3cuLl682IsVd8+ZM2cM7vPgwYMAgF/96lcdHmNNn2tdXR0iIiKwZcuWdrf/6U9/wttvv42tW7ciNTUVzs7OmDlzJhoaGjo85+7du5GQkID169cjIyMDERERmDlzJkpLS011G11yt3utr69HRkYGXnnlFWRkZOCzzz5DZmYmZs+e3el5jfku9JbOPlcAiIuLM6j7008/ves5LfVzBTq/3zvvs7i4GElJSRAEAfPmzbvreS3ts+3Kb83q1avx5ZdfYs+ePTh27BiKiorw6KOP3vW83fmeG00kq1VaWioCEI8dO9bhPh999JHo5ubWe0X1oPXr14sRERFd3v/xxx8XH374YYN10dHR4vLly3u4MtN78cUXxbCwMFGn07W73Zo/VwDif/7zH/2fdTqdqFQqxY0bN+rXVVVViXK5XPz00087PM/48ePFlStX6v+s1WpFf39/MTEx0SR1d8fP77U9aWlpIgAxLy+vw32M/S6YQ3v3unjxYnHOnDlGnccaPldR7NpnO2fOHPH++++/6z7W8Nn+/LemqqpKdHR0FPfs2aPf5/LlyyIAMSUlpd1zdPd7biw+qbFiarUaAODp6XnX/WpraxEcHIzAwEDMmTMHly5d6o3yekRWVhb8/f0RGhqKhQsXIj8/v8N9U1JSEBsba7Bu5syZSElJMXWZPaqpqQk7d+7E008/fdfJV635c73TtWvXoFKpDD47Nzc3REdHd/jZNTU1IT093eAYiUSC2NhYq/u81Wo1BEGAu7v7Xfcz5rtgSY4ePQofHx+Eh4djxYoVqKio6HBfW/pcS0pKsH//fixdurTTfS39s/35b016ejqam5sNPqchQ4YgKCiow8+pO9/z7mCosVI6nQ4vvfQS7rvvPowYMaLD/cLDw5GUlIS9e/di586d0Ol0mDhxIgoLC3ux2u6Jjo7G9u3bceDAAbz33nu4du0aJk+ejJqamnb3V6lU8PX1NVjn6+sLlUrVG+X2mM8//xxVVVV46qmnOtzHmj/Xn2v9fIz57MrLy6HVaq3+825oaMCaNWuwYMGCu04AaOx3wVLExcXh448/RnJyMv74xz/i2LFjmDVrFrRabbv728rnCgA7duyAi4tLp69kLP2zbe+3RqVSQSaTtQnid/ucuvM97w67maXb1qxcuRIXL17s9N1rTEwMYmJi9H+eOHEihg4divfffx9vvPGGqcu8J7NmzdL/86hRoxAdHY3g4GD861//6tJ//VirDz/8ELNmzYK/v3+H+1jz50otmpub8fjjj0MURbz33nt33ddavwtPPPGE/p9HjhyJUaNGISwsDEePHsWMGTPMWJnpJSUlYeHChZ024Lf0z7arvzWWgk9qrNCqVauwb98+HDlyBP379zfqWEdHR4wZMwbZ2dkmqs503N3dMXjw4A5rVyqVbVrfl5SUQKlU9kZ5PSIvLw+HDh3CM888Y9Rx1vy5tn4+xnx23t7ekEqlVvt5twaavLw8HDx48K5PadrT2XfBUoWGhsLb27vDuq39c2114sQJZGZmGv09Bizrs+3ot0apVKKpqQlVVVUG+9/tc+rO97w7GGqsiCiKWLVqFf7zn//g8OHDGDBggNHn0Gq1uHDhAvz8/ExQoWnV1tYiJyenw9pjYmKQnJxssO7gwYMGTzQs3UcffQQfHx88/PDDRh1nzZ/rgAEDoFQqDT676upqpKamdvjZyWQyREZGGhyj0+mQnJxs8Z93a6DJysrCoUOH4OXlZfQ5OvsuWKrCwkJUVFR0WLc1f653+vDDDxEZGYmIiAijj7WEz7az35rIyEg4OjoafE6ZmZnIz8/v8HPqzve8u8WTlVixYoXo5uYmHj16VCwuLtYv9fX1+n2efPJJ8fe//73+z6+99pr4zTffiDk5OWJ6err4xBNPiAqFQrx06ZI5bsEo//Vf/yUePXpUvHbtmvjdd9+JsbGxore3t1haWiqKYtt7/e6770QHBwfxz3/+s3j58mVx/fr1oqOjo3jhwgVz3YJRtFqtGBQUJK5Zs6bNNmv/XGtqasTvv/9e/P7770UA4qZNm8Tvv/9e3+Nnw4YNoru7u7h3717x/Pnz4pw5c8QBAwaIN2/e1J/j/vvvF9955x39n3ft2iXK5XJx+/bt4o8//ig+++yzoru7u6hSqXr9/u50t3ttamoSZ8+eLfbv3188d+6cwfe4sbFRf46f32tn3wVzudu91tTUiL/97W/FlJQU8dq1a+KhQ4fEsWPHioMGDRIbGhr057CWz1UUO//3WBRFUa1Wi05OTuJ7773X7jms4bPtym/Nc889JwYFBYmHDx8Wz549K8bExIgxMTEG5wkPDxc/++wz/Z+78j2/Vww1VgRAu8tHH32k32fq1Kni4sWL9X9+6aWXxKCgIFEmk4m+vr7iQw89JGZkZPR+8d0wf/580c/PT5TJZGJAQIA4f/58MTs7W7/95/cqiqL4r3/9Sxw8eLAok8nE4cOHi/v37+/lqrvvm2++EQGImZmZbbZZ++d65MiRdv/dbb0nnU4nvvLKK6Kvr68ol8vFGTNmtPl7CA4OFtevX2+w7p133tH/PYwfP148ffp0L91Rx+52r9euXevwe3zkyBH9OX5+r519F8zlbvdaX18vPvjgg2K/fv1ER0dHMTg4WFy2bFmbcGItn6sodv7vsSiK4vvvvy/26dNHrKqqavcc1vDZduW35ubNm+JvfvMb0cPDQ3RychJ/+ctfisXFxW3Oc+cxXfme3yvh1oWJiIiIrBrb1BAREZFNYKghIiIim8BQQ0RERDaBoYaIiIhsAkMNERER2QSGGiIiIrIJDDVERERkExhqiIiIyCYw1BAREZFNYKghIiIim8BQQ0RERDaBoYaIiIhswv8HZAHOkV8y0PIAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "\n",
    "# 获取数据集\n",
    "batch_size = 256\n",
    "train_dataLoader, test_dataLoader = load_data_fashion_minist(batch_size)\n",
    "\n",
    "# 定义网络\n",
    "net = nn.Sequential(\n",
    "    nn.Flatten(),               # 256*28*28 展开成 256*784(多用在全连接层之前)\n",
    "    nn.Linear(784, 10)\n",
    ").to(device)\n",
    "\n",
    "# 初始化各个层的参数\n",
    "def init_weights(m):\n",
    "    if type(m) == nn.Linear:\n",
    "        nn.init.normal_(m.weight, std=0.01)\n",
    "net.apply(init_weights)\n",
    "\n",
    "# 定义损失函数\n",
    "loss = nn.CrossEntropyLoss()\n",
    "\n",
    "# 训练策略\n",
    "trainer = torch.optim.SGD(net.parameters(), lr=0.1)\n",
    "\n",
    "# 开始训练\n",
    "losses = []\n",
    "epoch_num = []\n",
    "epochs = 20\n",
    "for epoch in range(epochs):\n",
    "    train_iter = iter(train_dataLoader)\n",
    "    test_iter = iter(test_dataLoader)\n",
    "\n",
    "    # 更新参数\n",
    "    for x, y in train_iter:\n",
    "        x,y = x.to(device), y.to(device)\n",
    "        l = loss(net(x), y)\n",
    "        trainer.zero_grad()\n",
    "        l.backward()\n",
    "        trainer.step()\n",
    "\n",
    "    # 计算测试集损失\n",
    "    test_loss = 0\n",
    "    for features, labels in test_iter:\n",
    "        features, labels = features.to(device), labels.to(device)\n",
    "        test_loss += loss(net(features), labels)\n",
    "    test_loss /= len(test_dataLoader)\n",
    "\n",
    "    # 添加数据\n",
    "    losses.append(test_loss.item())\n",
    "    epoch_num.append(epoch+1)\n",
    "\n",
    "plt.plot(epoch_num, losses)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 在 PyTorch 中，net.apply(func) 方法的作用是将函数 func 应用到神经网络 net 中的每一层。具体步骤如下：\n",
    "  - 遍历 net 中的每个模块（每一层）。\n",
    "  - 对每个模块调用函数 func，并将当前模块作为参数传递给 func。\n",
    "- 在 init_weights 函数中，参数 m 是由 PyTorch 自动传递的当前层（模块）对象。\n",
    "- `nn.init.normal_(tensor, mean=0.0, std=1.0)`"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "torch",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
